People can learn a new concept and use it compositionally, understanding how to blicket twice after learning how to blicket. In contrast, powerful sequence-to-sequence (seq2seq) neural networks fail such tests of compositionality, especially when composing new concepts together with existing concepts. In this paper, I show how memory-augmented neural networks can be trained to generalize compositionally through meta seq2seq learning. In this approach, models train on a series of seq2seq problems to acquire the compositional skills needed to solve new seq2seq problems. Meta se2seq learning solves several of the SCAN tests for compositional learning and can learn to apply implicit rules to variables.

If the nuances aren't appropriately fleshed out, then the argument unfortunately reads as a strawman that the subsequent work sets aflame. Similarly, in the results and discussion the authors are encouraged to write a bit more to contextualize what is happening: through meta-learning, the models are learning *the fact that* primitives that are seen only once/a few times should be treated the same way as others, which is impossible to learn without meta-learning. Again, this is also a potential explanation for how and why humans behave this way; they have a ton of experience learning such facts. The implications of these results should be looped back into the claims made from previous work, such as the paper that introduced the SCAN dataset. Finally, I think that in their argumentation the authors need to keep in mind that they are demonstrating that when a neural network is given an opportunity to learn X, then it will most likely learn X. This is a bit obvious, but too many within this debate lose sight of this fact while trying to argue that neural networks as a class of model are inherently deficient. The results presented here speak directly against this notion and deserve to be emphasized.

The reviewers agree that this paper is sound and of potential interest to some audiences. They disagree about whether the proposed solution (meta learning based on a large augmented data set) makes the result boring or interesting. Given that the results appear sound and are likely to yield interesting discussions at the conference, as they have among the reviewers, I recommend including it. I hope the authors revise the framing and discussion after taking into account the reviewers' comments, especially R1.

People can learn a new concept and use it compositionally, understanding how to "blicket twice" after learning how to "blicket." In contrast, powerful sequence-to-sequence (seq2seq) neural networks fail such tests of compositionality, especially when composing new concepts together with existing concepts. In this paper, I show how memory-augmented neural networks can be trained to generalize compositionally through meta seq2seq learning. In this approach, models train on a series of seq2seq problems to acquire the compositional skills needed to solve new seq2seq problems. Meta se2seq learning solves several of the SCAN tests for compositional learning and can learn to apply implicit rules to variables.

People can learn a new concept and use it compositionally, understanding how to "blicket twice" after learning how to "blicket." In contrast, powerful sequence-to-sequence (seq2seq) neural networks fail such tests of compositionality, especially when composing new concepts together with existing concepts. In this paper, I show how memory-augmented neural networks can be trained to generalize compositionally through meta seq2seq learning. In this approach, models train on a series of seq2seq problems to acquire the compositional skills needed to solve new seq2seq problems. Meta se2seq learning solves several of the SCAN tests for compositional learning and can learn to apply implicit rules to variables. Papers published at the Neural Information Processing Systems Conference.